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Purpose 


The Publish Interface may be used by Galleon system components to post data that is ready for further 
processing or user consumption. 

The interface provides an abstraction to the chain of tools used to process, store, and disseminate data 
collected or generated by the system. 

Design 


The Publish Interface is designed to allow Galleon system components to post finished data to the 
system along with context that allows for further processing or dissemination. 

Publish Process 

The Publish Interface processes published data using the following procedure. 

1. A Client produces data that should be further processed, analyzed, or stored. 

2. The Client calls the Post Handler of the Publish Interface, providing the source and type of the data 
being posted and the paths to one or more files and directories. 

3. The Publish Interface forwards the published data to one or more Ingesters through their Post 
Handlers. The interface identifies the appropriate ingesters using the source and type parameters. 

4. An Ingester processes, forwards, or stores the posted data. The ingester may resubmit its results to 
the Publish Interface for further processing, but must update the source and/or type parameter. 
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FIGURE 1: PUBLICATION OF DATA VIA THE PUBLISH INTERFACE 
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Ingesters 

Ingesters are system components that have registered a Post Handler with the Publish Interface. 
Ingesters may be used to process, analyze, or store data that has been collected or generated by the 
system. 

Source Tags 

The source of the data submitted to the Publish interface is identified using a Source Tag. An ingester 
will identify sources from which it would like to receive data during registration. 

The source tag can be used to provide the name of a tool and unique identifiers for tool instances. 

Type Tags 

The type of data submitted to the Publish interface is identified using a Type Tag. An ingester will 
identify types of data which it would like to receive during registration. 

The type tag can be used to specify the format of the data. 

Publishing Pipeline 

When data is posted to the Publish Interface, the data will be routed to one or more ingesters based on 
its source and type tags. These ingesters may be used to process that data to produce new data types, 
disseminate that data to external systems, or store that data in a type-specific location. 

Ingesters may resubmit data to the Publish Interface for further processing, but must not use the same 
source and type tags that were provided to it. 

When the Publish Interface or ingester has finished with the posted data it will return to its caller, 
allowing the interface client to clean up. The Post Handlers will return one of several error codes to the 
client. The caller may handle this error as appropriate, including updating a log and/or sending an error 
to the user. 

Layers 

The Publish Interface exposes Infrastructure Layer functionality related to the way the system stores and 
ingests data. A Post Handler is provided as part of the Publish Interface. 

Ingesters that have registered a Post Handler with the interface may be located in any layer. 

Implementation 


Configuration 

A system implementing the Publish Interface should define the following keys within the system 

configuration file: 

interface. publish, version The version of the Publish Interface that has been implemented by the 

system. The current version is T. Only one version key may be defined 
at a time. 

interface. publish. post The Post Handler for the Publish Interface. Only one post key may be 
defined at a time. 
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Handlers 

The Publish Interface defines one type of handler for posting data to the Interface or to an Ingester. 

Post Handler 

Post Handlers are provided by the interface implementation and any data ingesters. The handlers are 
used to post data to the processing pipeline and advance it through the pipeline. 

Behavior 

An interface client or ingester calls the Publish Interface's Post Handler to submit data to the publication 
pipeline. The interface will forward the operation to one or more ingesters based on the source and type 
tags. 

The format of the files and directories passed to an ingester's Post Handler are specific to the type tag 
associated with that data. 

The Post Handlers will not modify any files or directories provided during an operation. The interface 
client is responsible for cleaning up those files after the Post Handler has returned. 

Usage 

post_handler <source_tag> <type_tag> <file_path>+ 

source_tag tag specifying the source of data posted to the handler 
type_tag tag specifying the type of data posted to the handler 

fiie_path one or more local paths to files or directories containing the data to be published 

Handler Return Codes 

The Publish Interface defines return codes that will be returned from the various interface handlers. 

The code values and their meanings are provided below: 

0 Success 

1 Unspecified Error 

2 Invalid Arguments 
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Appendix A: Version History 


Date 

Description 

8 Nov 2013 

Version 1, Revision 1 

Publish Interface initialized and submitted for design review. 

28 Jul 2014 

Version 1, Revision 2 

Publish Interface revised after first Galleon pilot. 

1 Dec 2014 

Version 1, Revision 3 

Publish Interface revised after second Galleon pilot. 

1 Jun 2015 

Version 1, Final 

Publish Interface finalized and delivered. 
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Appendix B: Future Work 


The following items have been identified as possible areas for future work: 

Expose Registration Function 

Provide a mechanism to programmatically register a Post Handler with the Publish Interface. 
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